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ABSTRACT 

School District 916 in St* Paul, Minnesota, a special 
district which provides vocational-technical learning experiences for 
high school and post-secondary students, developed an Instructional 
Productivity System (IPS) • The IPS was designed to allow the district 
to monitor the success of the learner-centered instructional model 
whose features included flexible scheduling, performance contracting, 
individualized instruction, and accountability* Working in 
cooperation with a consulting firm, Educational Management Services, 
project components were developed to: 1) assess students 1 knowledge 
and skills upon entering vocational training; 2) measures students 9 
actual learning progress against expected progress; and 3) implement 
a computerized system to monitor student progress and to correlate 
this information with program costs and effectiveness data. Reports 
generated by the system include monthly reports of graduates, 
attendance reviews, student progress reports, and program progress 
reports. (PB) 
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BACKGROUND 

School District 916 



^ Considerable emphasis is being placed on accountability and efficiency 

in education. However, in spite of this emphasis, there does not appear to 
exist a comprehensive operating instructional information system that 
0 provides a means by which educational performance can be meaningfully 

t-iJ assessed. Special Intermediate School District 916, a recently established 

area vocational -technical institute in suburban St. Paul, Minnesota, has 
made considerable progress in developing a vocational instructional 
data system. Dr. William C. Knaak, the Superintendent of District 916, 
has conceived and initiated the development. 

The district has a commitment to providing and coordinating vocational- 
technical learning experiences for high school, post-high school, and adult 
students from eight member elementary-secondary school districts. It is 
conmitted to provide these experiences at a quality and cost level competitive 
with other public and private training institutions and agencies. Through 
advisory committees and cooperative programs with industry, the district 
strives to keep its occupational training programs highly relevant to 
occupational needs. 

To achieve its goals, the district has selected an alternative to 
teacher-centered modes of instruction. This alternative is a learner- 
centered instruction model involving flexible program scheduling, perform- 
ance contracting, and individualized instruction. In this mode, individual- 
ized learning packages (units of competency) permit each learner to progress 
at his own rate and to gain advance placement for those competencies he can 
demonstrate prior to beginning the learning activity. In addition to 
adjusting the student rate of learning, the packages are designed to acconmodate 
the student's style of learning and give him more than one option to achieve 
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a specific performance objective. The packages define what is to be 
learned and under what conditions and what degree of skill is required. 
Learner-centered instruction has been implemented at District 916 
with a diverse student population, consistent with their "fail -safe" 
learning environment. 

Quality of Education Act 

The 1971 Minnesota legislature passed a bill to create a Council 
on Quality Education. The council funded 32 innovative educational 
projects during the biennium. The projects are designed to "respond to the 
education needs of public school students, the professional growth and 
satisfaction of the school staffs, the effectiveness and efficiency of 
present schools in their learning processes, continuing pupil unit cost 
escalation and the resultant financial crises which this brings about." 

School District 916 received a grant from the Council to develop 
and implement its "Instructional Productivity System". As such, the 
system is consistent with the Council's accountability emphasis. The 
system was to develop procedures through which the district can 
derive the effectiveness of its instructional system by the design, 
development and implementation of program competency and instructional 
productivity modules. It is intended that the instructional efficiency 
information be utilized in establishing cost effectiveness and serve 
as criteria upon which to base decisions relating to instructional methods, 
media, equipment and facilities. 

Project Approach 

To supplement its own staff in the development of the system, District 
916 contracted with Educational Management Services, Inc. (EMS) of Minneapolis. 
EMS is a firm which provides consulting services in a wide range of 
education-related areas. One full-time and three part-time EMS staff 
members were assigned to the project They provided technical assistance 
to the district staff on project activities. 

Project activities were related to one of three project components. 
They were: 1) the selection and implementation of instruments and 
procedures for assessing the students 1 level of knowledge and vocational 
competency upon enrollment; 2) the design, development and implementation 
of procedures for assessing the actual learning progress of enrolled 
students in comparison with his expected progress; and 3) a computer-based 
system for monitoring student progress as well as for deriving program 
unit costs and related effectiveness information. 

ASSESSMENT COMPONENT 
General Enrollment Assessment 

Enrollment Assessment takes place after the students have been admitted 
into the vocational program being offered by District 916 on a first-come 
first-served basis. The purpose of enrollment assessment is to gather 
information to assess potential factors which might interfere with a student's 
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learning. If such factors are detected, the information 1s further used 
to determine how these learning impediments might be removed* 

Through the use of an assessment advisory committee which was 
representative of all sectors of the 916 instructional staff, a model 
for enrollment assessment and program readiness facilitation was developed. 
The model calls for a general assessment of the student's computations, 
communications, academic ability, and attitude upon entry into the 
school. If a student shows a deficiency in any of these areas, he is 
referred to the instructional support center for further diagnosis. 
Upon identification, he is then either sent directly to his instructional 
program or to a remedial program in the basic skills area or to school 
psychologists and/or outside community agencies that assist staff with 
more difficult problems. 

The district is in the process of establishing norms for its 
assessment instruments and attempting to determine appropriate cut offs 
for students in the various programs. 

Specific Program Assessment 

The purpose of program assessment is to determine when students have 
mastered the requisite cognitive, affective, and psychomotor objectives for 
a particular unit or learning package within their program. As a first step 
in the development of individualized learning systems, tasks for a particular 
occupation are determined. After the analysis, the indvidual learning 
packages are developed. Specifically, the packages consist of: 1) a defined 
task; 2) a terminal performance objective; 3) one or more micro-performance 
objectives; 4) learning steps; 5) resources; and 6) criterion exams 
and/or performance checkl ists. The criterion exams represent the "knowing 11 
assessment for each package; whereas, the performance checklists represent 
the "doing 11 assessment. The items on the exam determine r: a student has 
mastered all the stated micro-performance objectives for the package. The 
same exam can be used for pre-test purposes in the event that a student 
wants to check out of a given learning activity. 

The items on the criterion exam are in multiple choice format with a 
number of items referenced to each micro-performance objective. This 
format was chosen for its examination appropriateness and objectivity 
resulting in accurate replication of scoring. 

Performance checklists are developed in two varieties. They are 
used to evaluate a performance through observation, observing the student 
while performing, or for evaluating a product after completion. The 
checklists list the procedural steps necessary to complete a task and 
enable the evaluator to determine whether the specific steps were completed 
satisfactorily at a specified acceptance level. 

Program supervisors establish item files for storing test items and 
are able to group the items to test a particular package or a larger unit 
of study. Information is kept on the student responses to these items 
and an item analysis is conducted. The file also is updated based upon 
item analysis information. 
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SYSTEMS DEVELOPMENT COMPONENT 



System Overview 

The purpose of this project component is to develop a computer- 
based system capable of accepting, reporting and processing student 
progress data in a format which can be interfaced with the program 
oriented budgeting and accounting systems to provide indicators of the 
program instructional effectiveness. The system consists of six operational 
modules or sub-systems . Each module of the system is composed of one or more 
computer programs which either restructure data of existing files or 
generate reports. 

Major inputs to the system include the student enrollment form which 
contains demographic, program and test information on each student. 
This form is printed on a continuous form paper. The computer generated 
form is used as a turnaround document to update the student data file. 
A timecard is kept by each student for each learning package that he 
completes. Upon completion, the timecard is given to the instructor 
who, in turn, validates it and submits it for processing. Data from the 
timecard are used to determine the amount of department instructional 
resources utilized by a student in mastering a particular package and 
subsequently for his entire program. 

The two major files created and maintained for the system are the 

program-unit file and the student master and trailer files. These files 

are merged and integrated with the accounting subsystem to produce a 
number of monthly reports. 

The reports generated by the system include: 1) a monthly report of 
graduates which is necessary to identify graduates as they complete their 
individualized programs; 2) an attendance report which is derived from the 
timecard input; 3) a student progress report which gives the status of 
each student on his program, as well as a comparison to other students 
in his program; and 4) a program progress report which gives the status 
for all students within each program in terms of completion. 



